Add CheaperInference provider - #7010
Open
aiapienthusiast wants to merge 21 commits into
Open
aiapienthusiast wants to merge 21 commits into
aiapienthusiast wants to merge 21 commits into
Conversation
CheaperInference is an OpenAI-compatible gateway that serves each request from one of several providers for the requested model, ranked by discount, speed or a balance of both, at or below the model maker's list price. Adds the provider entry plus 16 relayed models, all using base_model against existing lab metadata with provider-side cost, cache rates, long-context tiers and reasoning_options only. Rates come from GET /v1/models (pricing_version sha256:d7fdd65d…, checked 2026-09-13T10:00:38Z). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Action items
|
The gateway documents that the reasoning field is forwarded to the serving provider but names no on/off control of its own, so only the effort levels inherited from the lab entries are authored. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Action items
|
Tested on the live chat completions endpoint: reasoning.enabled = false removes reasoning tokens and the reasoning field on kimi-k3, qwen-3-8-max, deepseek-v4.1-flash and deepseek-v4-flash-0731, and true restores them, so those four carry toggle with the wire path as a leading comment. On claude-sonnet-5 neither reasoning.enabled nor thinking had any observable effect on this path, so it stays effort-only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Registers a sync module for the CheaperInference provider so the hourly workflow keeps its rates current instead of leaving them to drift. The gateway's catalog endpoint is authoritative for pricing: exact per-million input, output, cache-read and cache-write rates, an above_threshold band with its own token threshold for models priced in two bands, and the limits this host actually serves. It exposes no reasoning controls, so the module sets skipCreates and refuses to sync a reasoning model whose reasoning_options were not hand-authored; image and video routes are skipped because their per-unit media pricing cannot be expressed by the cost schema. Reading the catalog needs a key, so the workflow gains a CHEAPERINFERENCE_API_KEY secret. The endpoint accepts a key restricted to read-only scope, which cannot spend from the account behind it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Author
|
Addressed all three.
|
Contributor
|
No actionable findings. |
github-actions
Bot
added
the
reviewer: ready
github-actions
Bot
removed
the
reviewer: ready
Contributor
Action items
|
Contributor
Action items
|
Contributor
Action items
|
Re-probed every model the review flagged on the live chat completions endpoint, with a unique prompt per request so nothing came from a cache. Reasoning comes back where the gateway catalog says it does not. claude-opus-4-8-fast returns reasoning_content on every plain request (7 of 7) and drops it for reasoning.enabled = false, and glm-4.6 returns reasoning_content with 128 to 592 reasoning tokens for every graded effort value. Both lose the reasoning = false line: opus-4-8-fast carries the verified toggle plus the lab effort set, glm-4.6 the toggle its first-party file also uses. gemini-3-5-flash reasons as well, and the gateway documents its own set in a 400 - minimal, low, medium, high, thinking that cannot be disabled - so it stays effort-only with those four values. reasoning.enabled = false and reasoning.effort = none both switch reasoning off on deepseek-v4-flash, deepseek-v4-pro and deepseek-v4-flash-0731, so each keeps a single off control, the toggle, with graded values and no none: high|xhigh, high|xhigh and low|high|max. minimax-m2.7 has a real toggle (no reasoning in 4 of 4 requests with reasoning.enabled = false, reasoning in every other form), so it replaces the empty options list. The four Claude routes the review asked about - opus 4.5, opus 4.6, sonnet 4.5 and sonnet 4.6 - return no reasoning_content and no reasoning token count at all, and effort, reasoning.enabled and thinking.budget_tokens are each accepted with 200 and change nothing. budget_tokens goes, and reasoning_options is empty rather than an effort list this host does not honour. qwen-3-8-max, re-verified as the review asked: reasoning.enabled = false no longer suppresses anything in 6 of 6 requests, so its toggle and effort list go the same way. The luna, sol and terra context tiers move from 271_999 to 272_000, the band start the sync module emits and gpt-6-astra already carries. claude-haiku-4.5, claude-opus-4.7, gemini-2.5-flash and gemini-3.1-flash-lite keep reasoning = false: the catalog declares them non-reasoning and five wire forms return nothing on each. bun run validate passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
qwen3-5-35b-a3b, qwen3-6-35b-a3b and qwen3.6-27b carried budget_tokens, but this gateway has no reasoning-budget field on the OpenAI-compatible path: reasoning.max_tokens = 64 and thinking.budget_tokens = 64 are both accepted with 200 and the reply still carries around a thousand characters of reasoning, the same as a plain request. Their toggles are re-verified and stay: reasoning.enabled = false returns no reasoning content on all three, and true restores it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Action items
|
Answers the third review round, all of it re-probed on the live endpoint with a unique prompt per request. reasoning = false replaces reasoning_options = [] on claude-opus-4.5, claude-opus-4.6, claude-sonnet-4.5 and claude-sonnet-4.6: no request form - plain, reasoning.enabled, reasoning.effort, thinking.budget_tokens - returns reasoning_content or a reasoning token count on those routes, so an empty options list would claim an always-on reasoner that is not there. The four that already carried reasoning = false keep it and now cite the probe in a leading comment. qwen-3-8-max keeps reasoning = true with an empty options list, because it does return reasoning_content on every request and neither the toggle nor effort = "none" suppresses it. interleaved records the side channel where the reply carries one: 33 models return reasoning_content, gemini-3-flash-preview returns reasoning_details, and glm-4.7 returns its reasoning in a plain reasoning field, which the schema covers as interleaved = true. The OpenAI-shaped routes count reasoning tokens without exposing any content, so they carry no interleaved. attachment is dropped on gemini-3-flash-preview and gemini-3.1-pro-preview, which accept an image_url part; the other sixteen reject it (400, or every upstream route failing) and now also override modalities.input to text only so the two agree. qwen-3-8-27b gets the toggle it actually forwards plus low|medium|xhigh and no none. gpt-5.5-pro is responses-shaped: /v1/chat/completions returns 400 and /v1/responses answers, where effort medium and high work. bun run validate passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Action items
|
Author
|
Both review rounds are answered here, and every claim below is a live probe on Reasoning: what this host actually returnsNine models return nothing on any wire form I could find - plain,
Three models the catalog reports as non-reasoning do reason, so
One off control per model
budget_tokens
interleavedAdded wherever the reply carries a side channel, left off where it does not. 33 models return attachment and modalitiesProbed with an Tiers
One thing the review did not ask about
Worth knowing for the sync module: the catalog's own |
…routes claude-fable-5, claude-fable-5.1 and claude-opus-5-fast advertise capabilities.vision true, but an image_url part is rejected on every attempt (502 or 400, "all available provider routes failed"), exactly like the other Anthropic routes on this host, so they get attachment = false and a text-only input modality. All three do return the reasoning side channel in reasoning_content, so interleaved stays. gemini-3.6-flash reports usage.completion_tokens_details.reasoning_tokens but the message carries only content on 8 of 8 probes, so no side channel is declared for it. muse-spark-1.2 returns its reasoning in reasoning_details on 3 of 3 untruncated probes (runs cut short by max_tokens carry none), so interleaved.field is set accordingly.
…he sync A model that reasons on this host but exposes no reasoning controls in the catalog still needs hand-authored reasoning_options. Throwing a plain Error aborted the whole run, so a single new model could silence the hourly sync for every other model. MissingReasoningOptionsError is the shape the runner already understands: it keeps the local file, records the ID and carries on, so the gap is reported without stopping the run.
…e reasoner The old assertion only matched the message text, so a plain Error would have passed it. It now asserts the error type and the model ID it carries. A second test drives syncProvider over a temporary models directory holding one researched and one unresearched reasoner, and checks what the run actually does with them: the unresearched file is left byte-for-byte alone, nothing is deleted, the other model still syncs, and the ID is handed to the missing-model issue opener.
Contributor
Action items
|
Author
|
All four items are addressed on the branch. 1. A single unresearched reasoner aborted the sync. 2-4. Probed against the host on 2026-09-16, same key and catalog version as the pricing lines:
Those three do return reasoning content, so Every one of these files carries the probe counts as a leading comment, so the next person reading it can see what was measured rather than what the catalog claims. |
The earlier image-input probe used a 1x1 PNG. Upstreams treat that as junk: routes that handle a normal image returned 400 or 502 "all available provider routes failed", and some that do accept images answered "White" for a red pixel. Every attachment and modalities value derived from that probe was wrong, in both directions. Re-probed with a 32x32 solid-colour PNG as a data: URI, four colours per model, max_tokens high enough that a reasoning model still answers, and a text-only control on every route that errored, so "no image input" is separated from "route down". A route counts as having image input only when it names the colour; several accept the request and then answer NOIMAGE or guess. Image input works on 36 of the 59 chat routes reachable on /v1/chat/completions, plus gpt-5.5-pro on the /v1/responses layer it requires. Each file states what was measured. Two notes: - deepseek-v4-flash-0731 reads images here although its base model is text-only (10 of 10 probes), so it carries an explicit attachment and modalities override rather than inheriting. - A remote https image URL is rejected by every route tested; only data: URIs work. Rates re-read at the same time (pricing_version sha256:6d2520b9, 2026-09-17T07:00:28Z): deepseek-v4.1-flash doubled, glm-5.3-flash and muse-spark-1.2 fell, gpt-5.4-nano rose. The other rates are unchanged.
Same 32x32 data: URI probe as the sibling routes: 4 of 4 colours named correctly, so this route keeps image input. Rates re-read at the same catalogue version.
Contributor
Action items
|
Author
|
Correction to my own comment above: the image-input part of it was wrong, and the branch now carries the fix. That probe used a 1x1 PNG. Upstreams treat a 1x1 image as junk, so routes that handle a normal image came back 400 or 502 "all available provider routes failed", and some that do accept images answered "White" for a red pixel. Every Re-probed today with a 32x32 solid-colour PNG as a Result: image input works on 36 of the 59 routes reachable on Two findings worth having in the repo:
Rates were re-read in the same pass (pricing_version |
19 tasks
…be notes Override-only rule: 13 text-only entries restated attachment = false and a [modalities] block identical to the lab entry, and 10 more restated modalities.output. Those lines are gone; base_model merges deeply, so the generated output is byte-identical (checked with bun run validate before and after). Headers: the 2026-09-16 no-image note is removed from the 8 files where the 2026-09-17 probe supersedes it - 7 that now verify image input, and claude-opus-4-8-fast, where it repeated the newer no-image note. bun run validate exits 0; bun test packages/core is 305 pass / 3 fail, the same 3 failures as on the branch without this change.
Contributor
Action items
|
…rwards The review is right: reasoning = false was wrong for six of the eight ids that carried it. Re-probed each id with six wire forms (plain, top-level reasoning_effort, reasoning.enabled, reasoning.effort, thinking.type with budget_tokens, thinking_budget), three samples per cell, scoring usage.completion_tokens_details.reasoning_tokens and message.reasoning_content. - claude-sonnet-4.5: toggle, thinking.type = enabled|disabled (205-308 reasoning tokens on, none off) - claude-sonnet-4.6: toggle plus effort medium|high|max (53 / 110-153 / 301-320) - claude-opus-4.6: toggle plus effort high|max (218-261 / 636-650 characters) - claude-opus-4.5 and claude-haiku-4.5: they reason, but no control could be reproduced - reasoning_options = [] - gemini-3.1-flash-lite: effort minimal|low|medium|high, and the host itself rejects reasoning_effort = "none" with "Gemini 3 thinking cannot be disabled. Supported reasoning effort values are minimal, low, medium, and high." - claude-opus-4.7 and gemini-2.5-flash keep reasoning = false: 18 requests each, no reasoning under any form The same host error also fixes three ladders that were authored from lab convention: gemini-3.1-pro-preview and gemini-3.7-flash were missing "minimal", and gemini-3.6-flash now carries the wire comment. claude-sonnet-5 drops to effort ["max"], the only level that measurably reasons there. bun run validate exits 0; bun test packages/core is 305 pass / 3 fail, the same 3 failures as on the branch without this change.
Author
|
Both review rounds are addressed, in 83d1fa5 and 838cd44. Override-only and stale headers (83d1fa5)13 text-only entries restated The 2026-09-16 no-image note is gone from the 8 files where the 2026-09-17 probe supersedes it, including the stacked pair on Reasoning controls (838cd44)The high-severity finding is correct, and
The host answers the Gemini 3 question itself.
That applies to every Gemini 3 id here ( On the low-severity item: One caveat worth recording, because it explains the earlier wrong values: the gateway picks an upstream route per request (the reply's
|
Contributor
Action items
|
claude-haiku-4.5: 21 more requests across all seven wire forms returned no reasoning at all, so
the empty options list goes and the id is reasoning = false. The two early reasoning replies are kept in
the header as an unreproducible route artefact.
claude-opus-4.5: a 34-request matrix separates the routes. On the route that answers as
anthropic/claude-opus-4-5, plain requests, reasoning_effort = "none" and reasoning.enabled = false all
return 0 reasoning tokens, while "low" returns 32, "medium" 32-105 and "high" 242-246, and
thinking.type = "enabled" returns 276-313. That is a toggle plus a graded effort ladder, so those are
authored and the empty list and the contradictory header claim are gone. The interleaved field goes too:
that route reports reasoning in usage and returns no reasoning_content.
claude-opus-4.7: 21 more requests, every one answered by Claude Platform on AWS, all with a reasoning
token count of 0. reasoning = false stands, on one route rather than a fan-out.
gemini-2.5-flash: 36 more requests, every one answered by Google AI Studio, all zero - including Google's
own controls in four shapes, among them
extra_body.google.thinking_config = { thinking_budget = 1024, include_thoughts = true }.
reasoning = false stands.
bun run validate passes; the generated JSON changes only for claude-haiku-4.5 and claude-opus-4.5.
Contributor
Action items
|
Picking the second shape the reviewer offered. reasoning.enabled is the on/off field, and it is the one control that survives the gateway's fan-out: true reasoned on 9 of 9 requests (193-237 reasoning tokens) and false returned 0 on 12 of 12, on both routes the id is served from. Requests with no reasoning field returned 0 on 8 of 9. reasoning_effort is graded but only on the route that answers as anthropic/claude-opus-4-5: "low" 32-33 (3 of 3), "medium" 32-105 (4 of 4), "high" 230-255 (5 of 5). The Amazon Bedrock route returns 0 for every effort value. "none" is removed from the list so it no longer overlaps the toggle, and the two wire paths are named in leading Toggle: / Effort: comments. bun run validate passes; the generated JSON changes only for claude-opus-4.5.
Author
|
Two rounds answered, in 6f74049 and c8414a7. Round 5 (6f74049)claude-haiku-4.5 — you were right that claude-opus-4.5 — the header really did contradict itself. Conditioning on the upstream route resolved it, and round 6 below settles the shape. claude-opus-4.7 — re-verified on a healthy route: 21 more requests, every one answered by Claude Platform on AWS, reasoning token count 0 throughout. gemini-2.5-flash — re-verified with Google's own controls in four shapes: Round 6 (c8414a7)Took your second option for A 30-request follow-up shows
So the file now reads: # Toggle: reasoning.enabled = true|false
# Effort: reasoning_effort = low|medium|highwith
|
Contributor
|
No actionable findings. |
github-actions
Bot
added
the
reviewer: ready
This comment was marked as duplicate.
This comment was marked as duplicate.
Re-read GET /v1/models: the pricing_version advanced from sha256:6d2520b9 to sha256:42cddd4d. Of the 60 Cheaper Inference entries, 57 are byte-for-byte unchanged and three moved, all through the discount rather than the model maker's list price: - deepseek-v4-flash: discount 45.09% -> 56.62%, so $0.071388/$0.142777 becomes $0.056398/$0.112796 - deepseek-v4.1-flash: 30.00% -> 59.72%, so $0.21/$0.84 becomes $0.120853/$0.483412 - muse-spark-1.2: 30.00% -> 15.00%, so $0.875/$2.975 becomes $1.0625/$3.6125 The Rates: header on those three files now cites the new pricing_version and timestamp; the other 57 keep theirs, because those figures were read at that version and have not changed. bun run validate passes; the generated JSON differs only for these three ids.
github-actions
Bot
removed
the
reviewer: ready
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cheaper Inference is an OpenAI-compatible gateway. It serves each request from one of several providers for the requested model. The
rankingfield selects discount, speed or a balance of both, withbalanceas the default. The price is at or below the model maker's list price.What this adds
providers/cheaperinference/withprovider.toml,logo.svgand 58 models. Each model isbase_model-ed against the existing lab entry. Each one carries onlycost,reasoning_optionsandlimitdeltas. The change adds no file under the top-levelmodels/.The 58 models are every chat model the gateway serves:
openai14,anthropic13,zhipuai9,google8,deepseek5,alibaba5, and one each fromxai,moonshotai,minimaxandmeta. The image-generation and video routes stay out.Rates
The rates come from
GET /v1/models, read at 2026-09-16T08:00:58Z,pricing_versionsha256:1099de72…. Cache rates are included.Four OpenAI models price a second band above 272k input tokens. The files record it as
[[cost.tiers]]:gpt-6-astra,gpt-5.6-sol,gpt-5.6-terraandgpt-5.6-luna.The gateway reprices often, so these rates are a timestamped snapshot. The first two comment lines of each file carry the source, the
pricing_versionand the discount off the maker's list price.Reasoning options
reasoning_optionscome from the lab entries. 51 of the 58 models have them.toggleis set on the 13 models where a live request confirms that the gateway forwards it. Each file carries a wire comment that names the exact field:{"reasoning": {"enabled": false}}onPOST /v1/chat/completionsfor 12 models;{"thinking": {"type": "disabled"}}forglm-4.5-air, which ignores the first form.In each case the disabled form returns no reasoning content, and the enabled form returns it.
claude-sonnet-5andqwen-3-8-maxkeep their other reasoning options, but notoggle.reasoning.enabledchanged nothing in the reply.gemini-2.5-flashandglm-4.6carry no reasoning options. The gateway reports no reasoning for them, although their lab entries have a toggle.Limits
limit.outputis set for every model, from the gateway's declared maximum.limit.contextis set only where the gateway's declared window differs from the lab entry (23 models).Daily sync
The provider is wired into the daily sync, so the snapshot does not go stale:
packages/core/src/sync/providers/cheaperinference.tsreads the same/v1/modelsendpoint. It reusesfactorBaseModelfrom the OpenRouter module, so every entry staysbase_model-ed.packages/core/src/sync/index.tsregisters the provider in theaggregatorsgroup..github/workflows/sync-models.ymlpasses aCHEAPERINFERENCE_API_KEYsecret.The endpoint needs a key to answer, so the sync run is a no-op until that secret exists. Nothing else in the workflow changes.
Checks
packages/core/test/cheaperinference.test.tscovers the module with 6 tests. They pass.bun validatepasses on the full tree.